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PAIRWISE INDEPENDENT SCHEDULING FOR NODES WITH A SINGLE 
PARAMETERIZABLE RADIO IN A WIRELESS NETWORK 

Background of the Invention 
5 [0001] A network where each node shares a single communication resource 
to exchange data between its (possibly many) neighbors needs to determine 
when and how long should it communicate with each of its neighbors so that 
there is a minimum of collision. This determination of timing and sequence of 
communication events is referred to here as the scheduling problem. As an 

10 instance of such a network we consider a mesh network where each node has 
one radio and is expected to exchange data with several other nodes by 
multiplexing its radio, the problem is for each node to create a schedule that 
determines when and for how long to exchange data with its neighbors to avoid 
collision. 

15 Qualifiers 

[0002] Widespread nodes: One reason to create a wireless mesh network is 
to provide widespread coverage by means of wireless nodes that form the 
backhaul infrastructure. This is an attempt to extend the model of the model of 
wired node access points where backhaul is over the wired network. In order to 
20 be successful the nodes are expected to be spread out in space, with the 

obvious requirement that at least one node connected to the existing mesh be 
able to communicate with a new outlying node. So the scheduling solution cannot 
rely on all the nodes being able to "hear" schedule announcements from one 
source. 

25 [0003] Directional radios: Directional radios allow the medium to be separated 
as transmissions on a beam towards one node does not overlap (significantly) 
and thus affect with a beam from the same radio on a different beam, or that of a 
beam possibly from a pair of nearby nodes. However this may prevent the use of 
broadcasts that are used by the 802.1 1 MAC for scheduling, or such broadcasts 

30 would negate some of the benefits of directional radios. 



Copy provided by USPTO from the IFW Image Database on 02/22/2005 



16892ROUS01P 



2 



[0004] The method proposed does not require scheduling signals to be 
duplicated — each node signals each of its neighbors exclusively, thereby 
exploiting directional nature of the antenna. 

[0005] Independent clocks and synchronization: One of the fundamental 
5 problems scheduling in time is that the notion of time in different entities, even if 
they have the same hardware, is non-uniform and varies with time. Furthermore, 
there isn't one channel or direct link through which all the nodes can obtain 
consistent timing and relaying this information over the nodes themselves is likely 
to be expensive. So following a pre-determined schedule will fail because of the 

10 relative drift of time in nodes. Some form of periodic synchronization of clocks 
may be required to deal with the drift between nodes. However due to different 
degrees of variations if one node synchronizes with one neighbor by adjusting its 
clock, it may in fact be de-synchronizing with another neighbor. Many 
approaches have been attempted to resolve these problems, including the 

15 following. 

[0006] Centralized scheduling: The 802.1 1 MAC [1] has its own scheduling 
algorithms. In the BSS mode of the algorithm, the Hybrid Control Point function in 
the access point determines the schedule based on the requests sent in by the 
stations and conveys this back to them. Each station then follows this schedule 

20 until the next time they receive another schedule. This method is only useful 
when all the stations can hear the schedules being broadcast by the AP. In a 
network using direction radios, such schemes will fail because the broadcasts will 
not reach nodes that are out of range or are not along the beam containing the 
transmitted schedules. The method described here however does not depend 

25 upon a single controller or upon a broadcast mode of operation. In fact, as long 
as a new node can communicate with one of the nodes already in the system, 
the new node can be incorporated and will be a part of the distributed schedule 
being maintained thorough the network. 

[0007] Decentralized scheduling: In 802.11 IBSS mode and in the Distributed 
30 Control Function mode of a BSS, each node independently attempts to send data 
after sensing its channel. This strategy allows networks with widely separated 
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nodes as there is no requirement for the broadcast of a central schedule. As the 
number of nodes and traffic increases on the medium, nodes that sense the 
medium to be busy will perform exponential back-off required for such random 
access of a shared medium and thus the throughput of the system will be much 
5 lower when compared to the method proposed here. 

[0008] Arbitration: Arbitration schemes that attempt to provide coordination 
such that two neighbors can exchange data when they have packets in their 
queues, such as RTS/CTS signaling may alleviate some of the overhead of 
exponential back-off. RTS/CTS schemes have been adapted to directional 

10 antennas [2][3], where parts or all of the signaling is done using directional 
antenna in order to try to encourage multiple simultaneous non-interfering 
transmissions [4]. However arbitration schemes like RTS/CTS incur costs that 
grow with the number of nodes in the network and setting up synchronization 
when a data packet to be sent out on a link arrives, may take an arbitrary amount 

15 of time depending upon the traffic serviced by the neighbors. The method 

proposed here is constant cost (independent of the number of nodes and their 
distribution) and provides bounded response time and does not require any back- 
off. In addition the proposed mechanism exploits the separation of media 
provided by directional antenna (and multiplexing other parameters) to obviate 

20 the RTS/CTS synchronization with a schedule that is guaranteed to be such that 
at no time sending receiving pair non-distinct. Given properly separated media 
this property of the schedule allows for interference less function without 
RTS/CTS or back-off overhead. 

[0009] Hierarchical: A master-slave relation like the BSS mode can be 
25 extended such that master's slave can be a master to other slaves, thereby 
forming a hierarchical tree of nodes [5]. Such a system can prevent back-off 
because schedules for each node will be determined by its corresponding 
master. However, the drawback of such a system is that the root of the whole 
tree or roots of sub-trees become points of failure, making all the subtending 
30 nodes vulnerable. In addition, propagating the schedule from the root to the 

leaves is likely to be cumbersome and susceptible to clock errors, where the time 
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reference in all the nodes will need to be guaranteed to be within bounds for the 
schedule to work. Otherwise if a node's time reference deviates, it will, for 
instance, start sending data when it's not supposed to and disrupt the system. 
The method described here has no masters that define the schedule for slaves. 
5 Indeed each node is independent to infer radio parameters to use from its 

environment and create its own schedule. This way, failure in the schedule of a 
node only affects its own schedule and the interaction with its neighbors, while 
interactions of the neighbors with their other peers are not affected. In addition, 
two peers keep the notion of their relative time within bounds, which incurs much 
10 less overhead than all nodes synchronizing to a wall clock determined by the root 
of the tree. 

[0010] The method described here does not require any node to ever change 
its own notion of time, hence avoiding de-synchronization with other nodes as 
described above. It relies on relative time intervals, using an arbitrary and 
15 coarse-grained notion of clock frequency ("ticks per second"), and attempts to 
bound the variation in such interval with each of its neighbors independently of 
others by periodically redetermining a fresh set of intervals, thereby avoiding drift. 

Summary of the Invention 

20 [0011] The present invention shows a non-interfering schedule of an ad hoc 
network of nodes with parameterizable radio, which can be achieved by nodes 
that independently coordinate to form time varying communicating pairs (that are 
guaranteed to be unique) by local message exchange. The present invention as 
described herein is a novel distributed algorithm that creates local schedules in 

25 such a way that the local schedule in each node and its corresponding data 

transmissions over the shared medium are non-interfering with the schedules of 
other nodes and their transmissions. The algorithm works by isolating pairs of 
nodes as communicating units and maintaining uniqueness of such pairs in time. 
This along with parameterized radio ensures that non-interfering node pairs in 

30 proximity as well as throughout the network can use the shared medium to 
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communicate at the same time without interfering with each other, obviate 
previously proposed collision avoidance schemes. 

[0012] The novelty of this method is the creation of node pairs, which are 
isolated using parameterized radio to simultaneously communicate over the 
5 shared medium and therefore do not need any other access control. In this way, 
it is expected that the nodes will be able to achieve high parallelism in data 
exchange, as well as maintaining robust and scalable scheduling. 
[001 3] Those skilled in the art will appreciate the scope of the present 
invention and realize additional aspects thereof after reading the following 
10 detailed description of the preferred embodiments in association with the 
accompanying drawing figures. 

Detailed Description of the Preferred Embodiments 

[0014] The embodiments set forth below represent the necessary information 
15 to enable those skilled in the art to practice the invention and illustrate the best 
mode of practicing the invention. Upon reading the following description in light 
of the accompanying drawing figures, those skilled in the art will understand the 
concepts of the invention and will recognize applications of these concepts not 
particularly addressed herein. It should be understood that these concepts and 
20 applications fall within the scope of the disclosure. 

[0015] Pairwise independent scheduling exploits radios that are 
parameterizable to be non-interfering (e.g. operate in different frequencies and/or 
use beamforming antenna to localize their area of interference) to provide a 
robust, scalable (in the number of nodes and distance) and adaptive distributed 
25 scheduling mechanism at a low, fixed overhead cost of periodic coordination 
between neighboring node pairs. 

Local scheduling to achieve robust inexpensive global scheduling 
[0016] Instead of creating a schedule that encompasses all nodes at once 
(e.g. a global schedule), or a cascading schedule determined by a local master 
30 for its slaves, the method described here decouples the network's schedule into 
independent schedules for neighboring node pairs. Each node is responsible for 
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its own schedule. The synchronization is between two neighbors and is 
independent of their synchronization with other neighbors. This allows nodes to 
form pairs that can communicate at the same time without collisions by exploiting 
the separation of the shared media afforded parameterizable radio. The schedule 
5 created is guaranteed to be such that at any time each sender-receiver pair is 
globally distinct in its constituent nodes from all other pairs, and by choosing 
suitable parameters for its radio each pair is made locally non-colliding (the same 
parameters can be used by other nodes that may be far away). The exact 
algorithm to choose such parameters is not described here. 

10 Independence from shared clocks purely interval based scheduling 

[0017] Pairwise independence in scheduling allows each node to maintain its 
own clock independently i.e., it doesn't need to change it's own clock to match 
the clock of any other node (e.g. master). The schedule of communication 
between two nodes works on time intervals, not external time or wall clocks. The 

15 start time and duration of the intervals are negotiated between the neighbors as 
required. They then independently count down to the event and if their clocks are 
within error bounds useful information exchange can occur. Note that this interval 
is such that two neighbors can synchronize to without altering their wall clocks. 
Each node also ensures that its previous commitments are not changed (due to 

20 any external synchronization or reset of wall clocks,) thereby atomically ensuring 
that subsequent commitments do not conflict with previous ones, thus providing 
non-preemptive scheduling. 

[0018] We use the following model to define the problem setting. There exists 
a set of nodes in the network N = {n 0 , r? f , ny}. Each node m can determine a 

25 set of it's neighbors, i.e. nodes that are first degree peers N B i = {n p , n q ,..} that it 
can communicate directly with. Upon initialization a node will attempt to gather 
information about its peers and try to communicate with them in order to create 
its neighbor set. The specifics of such a method to discover neighbors is beyond 
scope. We assume that after a node pair has established themselves as 

30 neighbors (i.e. added themselves to the others neighbor set) the two will be able 
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to communicate for some time sufficient to exchange a small constant number of 
messages. 

[0019] Time in a node is a non-decreasing counter, which can be read at any 
instant in a variable T. This counter can be viewed as slotted, where a time slot is 
5 an atomic time unit for the scheduler (i.e. a singular useful function can be 

accomplished in a time slot). A node's schedule is then a set of contiguous slots 
SC = {So , Si ,S 2 , Sk} = {to , ti , f 2 ... , }, where t 0 is the start of the next slot as 
well as the end of the present slot; U is the end of slot 0 and start of slot 1 and so 
on and slot S, occurs before Sj implies that the corresponding start time values 
10 are such that tg < tj. Furthermore the schedule only has entries for the finite future 
(i.e., t k is a finite value of the time counter) and the current value of Tis never 
greater than to. We also assume that a time constant of D units is known to be 
the upper bound for the drift in the clocks of the nodes. 

[0020] Each node creates a schedule similar to the one above, such that, if in 

15 a node n, the schedule for any time instant indicates that it is to communicate 

with a neighbor nj then there is a corresponding entry for the same time instant in 
the schedule of nj that indicates that it is to communicate with n$. If time was 
denoted in the schedule in terms of contiguous discrete intervals then the 
requirement above can be extended such that if in a node n„ the schedule for a 

20 time slot indicates that it is to communicate with a neighbor nythen there is a 

corresponding entry for the same time slot in the schedule of r?y that indicates that 
it is to communicate with nL Such time slots are represented as S, = {tj , nj}, 
where t t is its start time i.e., the value of Tat which slot / is to start, and n t is the 
corresponding neighbor or null for the times when there are no neighbor to 

25 communicate with. 

[0021] When a new node starts, its SC will have slots with null neighbors. As 
the nodes discover neighbors and exchange messages with them (to begin with 
this need not be coordinated in time, e.g. random overlap) they reserve slots in 
each others schedules, allowing subsequent scheduled negotiations to be 

30 coordinated in time. The duration of a slot that is negotiated is proportional to the 
time that is required to exchange the buffered data destined for a neighbor. Since 
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the slot will begin a finite amount of time after the exchange to reserve the slot, a 
function that computes the expected data to be exchanged using the present 
queue length and history is used at the time of negotiating the slot size. 
[0022] A message exchange sequence is used to negotiate the free slots and 
5 the time required for the nodes to drain their data. Negotiation between two 
nodes A and B is as follows. First, A sends S, T A the time it needs to drain its 
buffered packets destined for B and F A , that is a self of slots that is free at that 
time in node A. B performs similar computation to calculate T& and Fa, and upon 
the receipt of As message it calculates T, the total time required by both A and B 

10 to drain their data buffers. It then matches the free slots of both A and B, to find 
common time intervals, i.e. slots (or parts of slots), from F A and Fb until all the 
free slots of one node are exhausted or all of T has been booked. The resulting 
set of slots, Bab, is said to be the set of booked slots for that negotiation and is 
conveyed back to node A so that it can book those slots in its schedule as well. 

15 The process has been depicted in Figure 1. 

[0023] Since B AB is computed out of F A and F B and only one booking occurs 
and completes at any given time, the bookings can be guaranteed serialized and 
thus to not conflict with any existing bookings. Note that in order to convey free 
slots the nodes need to use relative time, because they have free running 

20 independent clocks. Relative time is calculated by a common event, referred to 
as the reference point of the negotiation. One such reference point is the 
reception of As message at B and if the MAC layer provides message 
acknowledgement the reference points for both A and B can be very close 
together in terms of absolute time. Therefore, F A and B A b are both denoted in 

25 terms of relative time, which can be calculated as time since the previous 

reference point. The further the reference point of B is from that of A, the more 
error will occur in the times booked at both ends. Also note that in the above 
discussion instead of sending time T A the amount of data can be directly 
communicated, if the other end can compute some time equivalent of it to 

30 calculate the total amount of time that needs to be booked in the free slots. 
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[0024] A given node can repeat this process periodically with all of its 
neighbors to continue to communicate with them. The message exchange above 
can be interleaved with user data exchange (e.g. as a preamble). However, 
scheduled communication with a neighbor may be lost if no booking for a future 
5 slot exists in the upcoming schedule. In order to prevent this from happening, 
the exchange to book future slots books, will in the absence of any data to be 
exchanged, book small slots that will be purely for the sake of completing another 
negotiation and thereby keeping the neighbor in the schedule. 
[0025] The mechanism above allows nodes to schedule data exchange with 

10 its neighbors, who will in turn be able to relay data to their neighbors and so forth. 
In this way, a large network can globally exchange data, hop-by-hop, by using a 
completely locally determined scheduler, where locality is a node and its direction 
neighbors. This method is independent of any global clocks and does not require 
communication with any entity other than direct neighbors. 

15 [0026] Those skilled in the art will recognize improvements and modifications 
to the preferred embodiments of the present invention. All such improvements 
and modifications are considered within the scope of the concepts disclosed 
herein. 
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TarTime requested by A 
Fa: Slots available at A 



Bab :Slots Booked for A<->B 



T(Ta, Tb) = T: Total Time for available A<->B 
BooHFsl, Fb) = Bab : Slots booked for A<->B 



Figure 1 
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